Enhancing precision in process conformance: stability, confidence and severity

Process Conformance is becoming a crucial area due to the changing nature of processes within an Information System. By confronting specifications against system executions (the main problem tackled in process conformance), both system bugs and obsolete/incorrect specifications can be revealed. This paper presents novel techniques to enrich the process conformance analysis for the precision dimension. The new features of the metric proposed in this paper provides a complete view of the precision between a log and a model. The techniques have been implemented as a plug-in in an open-source Process Mining platform and experimental results witnessing both the theory and the goals of this work are presented.Postprint (published version

Conformance checking and diagnosis in process mining

In the last decades, the capability of information systems to generate and record overwhelming amounts of event data has experimented an exponential growth in several domains, and in particular in industrial scenarios. Devices connected to the internet (internet of things), social interaction, mobile computing, and cloud computing provide new sources of event data and this trend will continue in the next decades. The omnipresence of large amounts of event data stored in logs is an important enabler for process mining, a novel discipline for addressing challenges related to business process management, process modeling, and business intelligence. Process mining techniques can be used to discover, analyze and improve real processes, by extracting models from observed behavior. The capability of these models to represent the reality determines the quality of the results obtained from them, conditioning its usefulness. Conformance checking is the aim of this thesis, where modeled and observed behavior are analyzed to determine if a model defines a faithful representation of the behavior observed a the log. Most of the efforts in conformance checking have focused on measuring and ensuring that models capture all the behavior in the log, i.e., fitness. Other properties, such as ensuring a precise model (not including unnecessary behavior) have been disregarded. The first part of the thesis focuses on analyzing and measuring the precision dimension of conformance, where models describing precisely the reality are preferred to overly general models. The thesis includes a novel technique based on detecting escaping arcs, i.e., points where the modeled behavior deviates from the one reflected in log. The detected escaping arcs are used to determine, in terms of a metric, the precision between log and model, and to locate possible actuation points in order to achieve a more precise model. The thesis also presents a confidence interval on the provided precision metric, and a multi-factor measure to assess the severity of the detected imprecisions. Checking conformance can be time consuming for real-life scenarios, and understanding the reasons behind the conformance mismatches can be an effort-demanding task. The second part of the thesis changes the focus from the precision dimension to the fitness dimension, and proposes the use of decomposed techniques in order to aid in checking and diagnosing fitness. The proposed approach is based on decomposing the model into single entry single exit components. The resulting fragments represent subprocesses within the main process with a simple interface with the rest of the model. Fitness checking per component provides well-localized conformance information, aiding on the diagnosis of the causes behind the problems. Moreover, the relations between components can be exploded to improve the diagnosis capabilities of the analysis, identifying areas with a high degree of mismatches, or providing a hierarchy for a zoom-in zoom-out analysis. Finally, the thesis proposed two main applications of the decomposed approach. First, the theory proposed is extended to incorporate data information for fitness checking in a decomposed manner. Second, a real-time event-based framework is presented for monitoring fitness.En las últimas décadas, la capacidad de los sistemas de información para generar y almacenar datos de eventos ha experimentado un crecimiento exponencial, especialmente en contextos como el industrial. Dispositivos conectados permanentemente a Internet (Internet of things), redes sociales, teléfonos inteligentes, y la computación en la nube proporcionan nuevas fuentes de datos, una tendencia que continuará en los siguientes años. La omnipresencia de grandes volúmenes de datos de eventos almacenados en logs abre la puerta al Process Mining (Minería de Procesos), una nueva disciplina a caballo entre las técnicas de gestión de procesos de negocio, el modelado de procesos, y la inteligencia de negocio. Las técnicas de minería de procesos pueden usarse para descubrir, analizar, y mejorar procesos reales, a base de extraer modelos a partir del comportamiento observado. La capacidad de estos modelos para representar la realidad determina la calidad de los resultados que se obtengan, condicionando su efectividad. El Conformance Checking (Verificación de Conformidad), objetivo final de esta tesis, permite analizar los comportamientos observados y modelados, y determinar si el modelo es una fiel representación de la realidad. La mayoría de los esfuerzos en Conformance Checking se han centrado en medir y asegurar que los modelos fueran capaces de capturar todo el comportamiento observado, también llamado "fitness". Otras propiedades, tales como asegurar la "precisión" de los modelos (no modelar comportamiento innecesario) han sido relegados a un segundo plano. La primera parte de esta tesis se centra en analizar la precisión, donde modelos describiendo la realidad con precisión son preferidos a modelos demasiado genéricos. La tesis presenta una nueva técnica basada en detectar "arcos de escape", i.e. puntos donde el comportamiento modelado se desvía del comportamiento reflejado en el log. Estos arcos de escape son usados para determinar, en forma de métrica, el nivel de precisión entre un log y un modelo, y para localizar posibles puntos de mejora. La tesis también presenta un intervalo de confianza sobre la métrica, así como una métrica multi-factorial para medir la severidad de las imprecisiones detectadas. Conformance Checking puede ser una operación costosa para escenarios reales, y entender las razones que causan los problemas requiere esfuerzo. La segunda parte de la tesis cambia el foco (de precisión a fitness), y propone el uso de técnicas de descomposición para ayudar en la verificación de fitness. Las técnicas propuestas se basan en descomponer el modelo en componentes con una sola entrada y una sola salida, llamados SESEs. Estos componentes representan subprocesos dentro del proceso principal. Verificar el fitness a nivel de subproceso proporciona una información detallada de dónde están los problemas, ayudando en su diagnóstico. Además, las relaciones entre subprocesos pueden ser explotadas para mejorar las capacidades de diagnóstico e identificar qué áreas concentran la mayor densidad de problemas. Finalmente, la tesis propone dos aplicaciones directas de las técnicas de descomposición: 1) la teoría es extendida para incluir información de datos a la verificación de fitness, y 2) el uso de sistemas descompuestos en tiempo real para monitorizar fitnes

Algorithms for Process Conformance and Process Refinement

Process Conformance is a crucial step in the area of Process Mining: the adequacy of a model derived from applying a discovery algorithm to a log must be certified before making further decisions that affect the system under consideration. In the first part of this thesis, among the different conformance dimensions, we propose a novel measure for precision, based on the simple idea of counting these situations were the model deviates from the log. Moreover, a log-based traversal of the model that avoids inspecting its whole behavior is presented. Experimental results show a significant improvement when compared to current approaches for the same task. Finally, the detection of the shortest traces in the model that lead to discrepancies is presented. In the second part of the thesis, two different approaches are proposed in order to use the precision analysis information for refining the model, improving its accuracy. The first one is based on the idea of break concurrencies reflected in the model but not in the log. The second one presents the Supervisory Control Theory as the mechanism to improve the accuracy of the models building supervisors for controlling the precision issues

Process Mining: descobrint models formals a partir de logs d’un sistema

L’objectiu d’aquest projecte és el desenvolupament d’una eina que faciliti certs aspectes del Process Mining. En concret, es pretén crear una aplicació que complementi el funcionament de l’eina Gene

Identidades fragmentadas y espacio público; la construcción social de los barrios en una ciudad turística de litoral, Puerto Vallarta Jal.

En este artículo se analizan cuatro factores que integran la identidad de los habitantes en los barrios o colonias de la ciudad turística y se retoma el valor de la calidad del espacio público. En los últimos años, de la mano de nuevos procesos económicos, políticos Y culturales, en muchos barrios se han producido cambios de fisionomía y de composición poblacional, de actividades, y con ello ciertos imaginarios han tendido a desaparecer a la vez que surgen otros nuevos. El espacio público destinado a las áreas de encuentro que han perdido el sentido de identidad por parte de la sociedad es por consecuencia un espacio que deja de ser considerado como lugar. Estos sitios conllevan a un abandono y al desperdicio del espacio físico en el cual se encuentran, propiciando aversión de los vecinos y visitantes a éstos espacios al mismo tiempo, se convierten en espacios degradados que la ciudadanía evita y elimina de su imaginario urbano. Dado este escenario, el presente trabajo intentara dilucidar a que remite esta “identidad barrial”, entendiendo que la pregunta por la génesis de este tipo de identidad resulta un paso previo imprescindible para luego poder analizar los discursos sobre su mutación o desaparición. Con este fin presentamos algunos de estos cambios operados por el mercado y el Estado y frente a los cuales, de acuerdo con los vecinos, la identidad barrial se ve damnificada, sobretodo en pos ya sea del abandono o de la recualificación de ciertos espacios públicos de cara al turismo. El espacio público es el lugar de encuentros e integración social, producto de las relaciones humanas, en el que se lleva a cabo un intercambio tanto social como cultural y recreativo. Tales intercambios, son parte de las necesidades que deberá de abastecer la ciudad a la sociedad que la habita con el fin de mantener a esta en un nivel de calidad de vida no solo aceptable, sino que óptimo. Sin embargo, dichas necesidades no siempre son bien abastecidas por la ciudad, al descuidar los espacios que con anterioridad han sido destinados para llevar a cabo actividades encauzadas a satisfacerlas y generar cohesión social.This article describes four factors that make up the identity of the people in the neighborhoods and colonies of the resort town and the value of the quality of public space is retaken. In recent years, with the help of new economic, political and cultural processes in many neighborhoods there have been changes of the physiognomy and population composition, activities, and thus certain imageries have tended to disappear while new ones emerge . The public space for meeting areas that have lost their sense of identity by society is consequently a space that is no longer considered as a place. These sites are abandonment and turn on a waste of physical space in which they are found, leading aversion of residents and visitors to these areas at the same time, become degraded areas that citizenship prevents and eliminates their urban imagery. Given this scenario, the present study attempted to elucidate forwards this "neighborhood identity", meaning that the question of the genesis of this type of identity is an essential first step to later analyze the discourses on its mutation or loss.Peer Reviewe

De la tradición a la revitalización urbana, una apuesta de estrategia para el reposicionamiento turístico: el centro tradicional de Puerto Vallarta, Jal. México

En este articulo se analizan brevemente tres elementos que caracterizan la problemática del reposicionamiento de las ciudades en general en el escenario mundial, y en particular de las ciudades turísticas de Latinoamérica, en específico el caso de la ciudad de Puerto Vallarta, Jalisco, México. Ciudad turística de litoral. En la primera parte se analizan algunos instrumentos de reposicionamiento que han adoptado como estrategia algunas ciudades en el mundo. En la segunda parte se hace una aproximación a los problemas que enfrentan las ciudades al procurar revitalizar zonas céntricas deterioradas. En la tercera parte se analiza el caso de la ciudad de Puerto Vallarta, Jalisco, México la cual en medio de un proceso de crisis turística ha adoptado en una primera instancia la revitalización urbana del centro tradicional de la ciudad, como una estrategia de reposicionamiento que le permitirá recuperar su principal ventaja comparativa; la imagen de “pueblo mexicano” con la que compite a nivel internacional en el mercado turístico. En Puerto Vallarta, el emprendimiento de esta estrategia, supone la posibilidad de iniciar una nueva etapa en el ciclo de vida de la ciudad como destino turístico maduro, es una ciudad en expansión con un desarrollo turístico consolidado. Sigue siendo el escenario de un crecimiento singular tanto económico como urbano en la costa occidente del país.Peer Reviewe

Conformance checking in UML artifact-centric business process models

Business artifacts have appeared as a new paradigm to capture the information required for the complete execution and reasoning of a business process. Likewise, conformance checking is gaining popularity as a crucial technique that enables evaluating whether recorded executions of a process match its corresponding model. In this paper, conformance checking techniques are incorporated into a general framework to specify business artifacts. By relying on the expressive power of an artifact-centric specification, BAUML, which combines UML state and activity diagrams (among others), the problem of conformance checking can be mapped into the Petri net formalism and its results be explained in terms of the original artifact-centric specification. In contrast to most existing approaches, ours incorporates data constraints into the Petri nets, thus achieving conformance results which are more precise. We have also implemented a plug-in, within the ProM framework, which is able to translate a BAUML into a Petri net to perform conformance checking. This shows the feasibility of our approach.Peer ReviewedPostprint (author's final draft

Process mining for healthcare: Characteristics and challenges

Process mining techniques can be used to analyse business processes using the data logged during their execution. These techniques are leveraged in a wide range of domains, including healthcare, where it focuses mainly on the analysis of diagnostic, treatment, and organisational processes. Despite the huge amount of data generated in hospitals by staff and machinery involved in healthcare processes, there is no evidence of a systematic uptake of process mining beyond targeted case studies in a research context. When developing and using process mining in healthcare, distinguishing characteristics of healthcare processes such as their variability and patient-centred focus require targeted attention. Against this background, the Process-Oriented Data Science in Healthcare Alliance has been established to propagate the research and application of techniques targeting the data-driven improvement of healthcare processes. This paper, an initiative of the alliance, presents the distinguishing characteristics of the healthcare domain that need to be considered to successfully use process mining, as well as open challenges that need to be addressed by the community in the future.This work is partially supported by ANID FONDECYT 1220202, Dirección de Investigación de la Vicerrectoría de Investigación de la Pontificia Universidad Católica de Chile - PUENTE [Grant No. 026/ 2021]; and Agencia Nacional de Investigación y Desarrollo [Grant Nos. ANID-PFCHA/Doctorado Nacional/2019–21190116, ANID-PFCHA/ Doctorado Nacional/2020–21201411]. With regard to the co-author Hilda Klasky, this manuscript has been authored by UT-Battelle, LLC, under contract DE-AC05-00OR22725 with the US Department of Energy (DOE). The US government retains and the publisher, by accepting the article for publication, acknowledges that the US government retains a nonexclusive, paid-up, irrevocable, worldwide license to publish or reproduce the published form of this manuscript, or allow others to do so, for US government purposes. DOE will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-public-accessplan).Peer ReviewedArticle signat per 55 autors/es: Jorge Munoz-Gama (a)* , Niels Martin (b,c)* , Carlos Fernandez-Llatas (d,g)* , Owen A. Johnson (e)* , Marcos Sepúlveda (a)* , Emmanuel Helm (f)* , Victor Galvez-Yanjari (a)* , Eric Rojas (a) , Antonio Martinez-Millana (d) , Davide Aloini (k) , Ilaria Angela Amantea (l,q,r) , Robert Andrews (ab), Michael Arias (z) , Iris Beerepoot (o) , Elisabetta Benevento (k) , Andrea Burattin (ai), Daniel Capurro (j) , Josep Carmona (s) , Marco Comuzzi (w), Benjamin Dalmas (aj,ak), Rene de la Fuente (a) , Chiara Di Francescomarino (h) , Claudio Di Ciccio (i) , Roberto Gatta (ad,ae), Chiara Ghidini (h) , Fernanda Gonzalez-Lopez (a) , Gema Ibanez-Sanchez (d) , Hilda B. Klasky (p) , Angelina Prima Kurniati (al), Xixi Lu (o) , Felix Mannhardt (m), Ronny Mans (af), Mar Marcos (v) , Renata Medeiros de Carvalho (m), Marco Pegoraro (x) , Simon K. Poon (ag), Luise Pufahl (u) , Hajo A. Reijers (m,o) , Simon Remy (y) , Stefanie Rinderle-Ma (ah), Lucia Sacchi (t) , Fernando Seoane (g,am,an), Minseok Song (aa), Alessandro Stefanini (k) , Emilio Sulis (l) , Arthur H. M. ter Hofstede (ab), Pieter J. Toussaint (ac), Vicente Traver (d) , Zoe Valero-Ramon (d) , Inge van de Weerd (o) , Wil M.P. van der Aalst (x) , Rob Vanwersch (m), Mathias Weske (y) , Moe Thandar Wynn (ab), Francesca Zerbato (n) // (a) Pontificia Universidad Catolica de Chile, Chile; (b) Hasselt University, Belgium; (c) Research Foundation Flanders (FWO), Belgium; (d) Universitat Politècnica de València, Spain; (e) University of Leeds, United Kingdom; (f) University of Applied Sciences Upper Austria, Austria; (g) Karolinska Institutet, Sweden; (h) Fondazione Bruno Kessler, Italy; (i) Sapienza University of Rome, Italy; (j) University of Melbourne, Australia; (k) University of Pisa, Italy; (l) University of Turin, Italy; (m) Eindhoven University of Technology, The Netherlands; (n) University of St. Gallen, Switzerland; (o) Utrecht University, The Netherlands; (p) Oak Ridge National Laboratory, United States; (q) University of Bologna, Italy; (r) University of Luxembourg, Luxembourg; (s) Universitat Politècnica de Catalunya, Spain; (t) University of Pavia, Italy; (u) Technische Universitaet Berlin, Germany; (v) Universitat Jaume I, Spain; (w) Ulsan National Institute of Science and Technology (UNIST), Republic of Korea; (x) RWTH Aachen University, Germany; (y) University of Potsdam, Germany; (z) Universidad de Costa Rica, Costa Rica; (aa) Pohang University of Science and Technology, Republic of Korea; (ab) Queensland University of Technology, Australia; (ac) Norwegian University of Science and Technology, Norway; (ad) Universita degli Studi di Brescia, Italy; (ae) Lausanne University Hospital (CHUV), Switzerland; (af) Philips Research, the Netherlands; (ag) The University of Sydney, Australia; (ah) Technical University of Munich, Germany; (ai) Technical University of Denmark, Denmark; (aj) Mines Saint-Etienne, France; (ak) Université Clermont Auvergne, France; (al) Telkom University, Indonesia; (am) Karolinska University Hospital, Sweden; (an) University of Borås, SwedenPostprint (published version

Genetic Footprints of Iberian Cattle in America 500 Years after the Arrival of Columbus

Background: American Creole cattle presumably descend from animals imported from the Iberian Peninsula during the period of colonization and settlement, through different migration routes, andmay have also suffered the influence of cattle directly imported from Africa. The introduction of European cattle, which began in the 18th century, and later of Zebu from India, has threatened the survival of Creole populations, some of which have nearly disappeared or were admixed with exotic breeds. Assessment of the genetic status of Creole cattle is essential for the establishment of conservation programs of these historical resources. Methodology/Principal Findings: We sampled 27 Creole populations, 39 Iberian, 9 European and 6 Zebu breeds. We used microsatellite markers to assess the origins of Creole cattle, and to investigate the influence of different breeds on their genetic make-up. The major ancestral contributions are from breeds of southern Spain and Portugal, in agreement with the historical ports of departure of ships sailing towards the Western Hemisphere. This Iberian contribution to Creoles may also include some African influence, given the influential role that African cattle have had in the development of Iberian breeds, but the possibility of a direct influence on Creoles of African cattle imported to America can not be discarded. In addition to the Iberian influence, the admixture with other European breeds was minor. The Creoles from tropical areas, especially those from the Caribbean, show clear signs of admixture with Zebu. Conclusions/Significance: Nearly five centuries since cattle were first brought to the Americas, Creoles still show a strong and predominant signature of their Iberian ancestors. Creole breeds differ widely from each other, both in genetic structure and influences from other breeds. Efforts are needed to avoid their extinction or further genetic erosion, which would compromise centuries of selective adaptation to a wide range of environmental condition

Battle of Postdisaster Response and Restoration

[EN] The paper presents the results of the Battle of Postdisaster Response and Restoration (BPDRR) presented in a special session at the first International water distribution systems analysis & computing and control in the water industry (WDSA/CCWI) Joint Conference, held in Kingston, Ontario, Canada, in July 2018. The BPDRR problem focused on how to respond and restore water service after the occurrence of five earthquake scenarios that cause structural damage in a water distribution system. Participants were required to propose a prioritization schedule to fix the damages of each scenario while following restrictions on visibility/nonvisibility of damages. Each team/approach was evaluated against six performance criteria: (1) time without supply for hospital/firefighting, (2) rapidity of recovery, (3) resilience loss, (4) average time of no user service, (5) number of users without service for eight consecutive hours, and (6) water loss. Three main types of approaches were identified from the submissions: (1) general-purpose metaheuristic algorithms, (2) greedy algorithms, and (3) ranking-based prioritizations. All three approaches showed potential to solve the challenge efficiently. The results of the participants showed that for this network, the impact of a large-diameter pipe failure on the network is more significant than several smaller pipes failures. The location of isolation valves and the size of hydraulic segments influenced the resilience of the system during emergencies. On average, the interruptions to water supply (hospitals and firefighting) varied considerably among solutions and emergency scenarios, highlighting the importance of private water storage for emergencies. The effects of damages and repair work were more noticeable during the peak demand periods (morning and noontime) than during the low-flow periods; and tank storage helped to preserve functionality of the network in the first few hours after a simulated event. (C) 2020 American Society of Civil Engineers.Paez, D.; Filion, Y.; Castro-Gama, M.; Quintiliani, C.; Santopietro, S.; Sweetapple, C.; Meng, F.... (2020). Battle of Postdisaster Response and Restoration. Journal of Water Resources Planning and Management. 146(8):1-13. https://doi.org/10.1061/(ASCE)WR.1943-5452.0001239S1131468Balut A. R. Brodziak J. Bylka and P. Zakrzewski. 2018. “Battle of post-disaster response and restauration (BPDRR).” In Proc. 1st Int. WDSA/CCWI 2018 Joint Conf. 14. Kingston Canada: Open Journal Systems.Bibok A. 2018. “Near-optimal restoration scheduling of damaged drinking water distribution systems using machine learning.” In Proc. 1st Int. WDSA/CCWI 2018 Joint Conf. 14. Kingston Canada: Open Journal Systems.Castro-Gama M. C. Quintiliani and S. Santopietro. 2018. “After earthquake post-disaster response using a many-objective approach a greedy and engineering interventions.” In Proc. 1st Int. WDSA/CCWI 2018 Joint Conf. 14. Kingston Canada: Open Journal Systems.Cimellaro, G. P., Tinebra, A., Renschler, C., & Fragiadakis, M. (2016). New Resilience Index for Urban Water Distribution Networks. Journal of Structural Engineering, 142(8). doi:10.1061/(asce)st.1943-541x.0001433Cover, T., & Hart, P. (1967). Nearest neighbor pattern classification. IEEE Transactions on Information Theory, 13(1), 21-27. doi:10.1109/tit.1967.1053964Creaco, E., Franchini, M., & Alvisi, S. (2010). Optimal Placement of Isolation Valves in Water Distribution Systems Based on Valve Cost and Weighted Average Demand Shortfall. Water Resources Management, 24(15), 4317-4338. doi:10.1007/s11269-010-9661-5Deb, K., Mohan, M., & Mishra, S. (2005). Evaluating the ε-Domination Based Multi-Objective Evolutionary Algorithm for a Quick Computation of Pareto-Optimal Solutions. Evolutionary Computation, 13(4), 501-525. doi:10.1162/106365605774666895Deuerlein J. D. Gilbert E. Abraham and O. Piller. 2018. “A greedy scheduling of post-disaster response and restoration using pressure-driven models and graph segment analysis.” In Proc. 1st Int. WDSA/CCWI 2018 Joint Conf. 14. Kingston Canada: Open Journal Systems.Deuerlein, J. W. (2008). Decomposition Model of a General Water Supply Network Graph. Journal of Hydraulic Engineering, 134(6), 822-832. doi:10.1061/(asce)0733-9429(2008)134:6(822)Di Nardo, A., Di Natale, M., Giudicianni, C., Santonastaso, G. F., & Savic, D. (2018). Simplified Approach to Water Distribution System Management via Identification of a Primary Network. Journal of Water Resources Planning and Management, 144(2), 04017089. doi:10.1061/(asce)wr.1943-5452.0000885Eliades D. G. M. Kyriakou S. Vrachimis and M. M. Polycarpou. 2016. “EPANET-MATLAB toolkit: An open-source software for interfacing EPANET with MATLAB.” In Proc. 14th Int. Conf. on Computing and Control for the Water Industry (CCWI) 8. The Hague The Netherlands: International Water Conferences. https://doi.org/10.5281/zenodo.831493.Fragiadakis, M., Christodoulou, S. E., & Vamvatsikos, D. (2013). Reliability Assessment of Urban Water Distribution Networks Under Seismic Loads. Water Resources Management, 27(10), 3739-3764. doi:10.1007/s11269-013-0378-0Gilbert, D., Abraham, E., Montalvo, I., & Piller, O. (2017). Iterative Multistage Method for a Large Water Network Sectorization into DMAs under Multiple Design Objectives. Journal of Water Resources Planning and Management, 143(11), 04017067. doi:10.1061/(asce)wr.1943-5452.0000835Hill, D., Kerkez, B., Rasekh, A., Ostfeld, A., Minsker, B., & Banks, M. K. (2014). Sensing and Cyberinfrastructure for Smarter Water Management: The Promise and Challenge of Ubiquity. Journal of Water Resources Planning and Management, 140(7), 01814002. doi:10.1061/(asce)wr.1943-5452.0000449Hwang, H. H. M., Lin, H., & Shinozuka, M. (1998). Seismic Performance Assessment of Water Delivery Systems. Journal of Infrastructure Systems, 4(3), 118-125. doi:10.1061/(asce)1076-0342(1998)4:3(118)Li Y. J. Gao C. Jian C. Ou and S. Hu. 2018. “A two-stage post-disaster response and restoration method for the water distribution system.” In Proc. 1st Int. WDSA/CCWI 2018 Joint Conf. 14. Kingston Canada: Open Journal Systems.Liu, W., Zhao, Y., & Li, J. (2014). Seismic functional reliability analysis of water distribution networks. Structure and Infrastructure Engineering, 11(3), 363-375. doi:10.1080/15732479.2014.887121Luong, H. T., & Nagarur, N. N. (2005). Optimal Maintenance Policy and Fund Allocation in Water Distribution Networks. Journal of Water Resources Planning and Management, 131(4), 299-306. doi:10.1061/(asce)0733-9496(2005)131:4(299)MacQueen J. B. 1967. “Some methods for classification and analysis of multivariate observations.” In Vol. 1 of Proc. 5th Berkeley Symp. on Mathematical Statistics and Probability 281–297. Berkeley: University of California Press.Mahmoud, H. A., Kapelan, Z., & Savić, D. (2018). Real-Time Operational Response Methodology for Reducing Failure Impacts in Water Distribution Systems. Journal of Water Resources Planning and Management, 144(7), 04018029. doi:10.1061/(asce)wr.1943-5452.0000956Meng, F., Fu, G., Farmani, R., Sweetapple, C., & Butler, D. (2018). Topological attributes of network resilience: A study in water distribution systems. Water Research, 143, 376-386. doi:10.1016/j.watres.2018.06.048Ostfeld, A., Uber, J. G., Salomons, E., Berry, J. W., Hart, W. E., Phillips, C. A., … Walski, T. (2008). The Battle of the Water Sensor Networks (BWSN): A Design Challenge for Engineers and Algorithms. Journal of Water Resources Planning and Management, 134(6), 556-568. doi:10.1061/(asce)0733-9496(2008)134:6(556)Paez D. Y. Filion and M. Hulley. 2018a. “Battle of post-disaster response and restoration (BPDRR)—Problem description and rules.” Accessed June 14 2019. https://www.queensu.ca/wdsa-ccwi2018/problem-description-and-files.Paez, D., Suribabu, C. R., & Filion, Y. (2018). Method for Extended Period Simulation of Water Distribution Networks with Pressure Driven Demands. Water Resources Management, 32(8), 2837-2846. doi:10.1007/s11269-018-1961-1Salcedo C. A. Aguilar P. Cuero S. Gonzalez S. Muñoz J. Pérez A. Posada J. Robles and K. Vargas. 2018. “Determination of the hydraulic restoration capacity of b-city involving a multi-criteria decision support model.” In Proc. 1st Int. WDSA/CCWI 2018 Joint Conf. 14. Kingston Canada: Open Journal Systems.Santonastaso G. F. E. Creaco A. Di Nardo and M. Di Natale. 2018. “Post-disaster response and restauration of B-town network based on primary network.” In Vol. 1 of Proc. 1st Int. WDSA/CCWI 2018 Joint Conf. Kingston Canada: Open Journal Systems.Sophocleous S. E. Nikoloudi H. A. Mahmoud K. Woodward and M. Romano. 2018. “Simulation-based framework for the restoration of earthquake-damaged water distribution networks using a genetic algorithm.” In Proc. 1st Int. WDSA/CCWI 2018 Joint Conf. 14. Kingston Canada: Open Journal Systems.Sweetapple C. F. Meng R. Farmani G. Fu and D. Butler. 2018. “A heuristic approach to water network post-disaster response and restoration.” In Proc. 1st Int. WDSA/CCWI 2018 Joint Conf. 14. Kingston Canada: Open Journal Systems.Tabucchi, T., Davidson, R., & Brink, S. (2010). Simulation of post-earthquake water supply system restoration. Civil Engineering and Environmental Systems, 27(4), 263-279. doi:10.1080/10286600902862615Taormina, R., Galelli, S., Tippenhauer, N. O., Salomons, E., Ostfeld, A., Eliades, D. G., … Ohar, Z. (2018). Battle of the Attack Detection Algorithms: Disclosing Cyber Attacks on Water Distribution Networks. Journal of Water Resources Planning and Management, 144(8), 04018048. doi:10.1061/(asce)wr.1943-5452.0000969Walski, T. M. (1993). Water distribution valve topology for reliability analysis. Reliability Engineering & System Safety, 42(1), 21-27. doi:10.1016/0951-8320(93)90051-yWang, Y., Au, S.-K., & Fu, Q. (2010). Seismic Risk Assessment and Mitigation of Water Supply Systems. Earthquake Spectra, 26(1), 257-274. doi:10.1193/1.3276900Yoo, D. G., Kang, D., & Kim, J. H. (2016). Optimal design of water supply networks for enhancing seismic reliability. Reliability Engineering & System Safety, 146, 79-88. doi:10.1016/j.ress.2015.10.001Zhang Q. F. Zheng K. Diao B. Ulanicki and Y. Huang. 2018. “Solving the battle of post-disaster response and restauration (BPDRR) problem with the aid of multi-phase optimization framework.” In Proc. 1st Int. WDSA/CCWI 2018 Joint Conf. 14. Kingston Canada: Open Journal Systems